Skip to content

[CI] Move nightly SYCL benchmarking into a separate workflow #19280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Jul 4, 2025
Merged

[CI] Move nightly SYCL benchmarking into a separate workflow #19280

merged 9 commits into from
Jul 4, 2025

Conversation

ianayl
Copy link
Contributor

@ianayl ianayl commented Jul 2, 2025

Performance of SYCL has regressed a lot recently, causing nightly to fail consistently. However, if nightly is constantly failing, people may start to ignore nightly. #18766 should help with the amount of failures, but I am still figuring out the best parameters to use here, and additionally often times regressions are caused by driver changes, which is out of intel/llvm's control. Thus, for future's sake, it might be best to move benchmarking out of sycl-nightly.yml.

This PR moves nightly benchmarking into its own workflow/nightly job, 1hr before sycl-nightly.yml, in order to reduce the amount of failures in sycl nightly.

@ianayl ianayl temporarily deployed to WindowsCILock July 2, 2025 17:34 — with GitHub Actions Inactive
@ianayl ianayl mentioned this pull request Jul 2, 2025
Merged
@ianayl ianayl requested a review from a team July 2, 2025 18:27
@ianayl
Copy link
Contributor Author

ianayl commented Jul 2, 2025

Hi dpcpp-devops-reviewers, can I get some thoughts/opinions on what I'm doing here?

If everyone is on-board with the changes being proposed here, we can go ahead with merging #19281 in order to test the changes being made here.

@ianayl ianayl temporarily deployed to WindowsCILock July 2, 2025 20:38 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 2, 2025 20:38 — with GitHub Actions Inactive
uditagarwal97 pushed a commit that referenced this pull request Jul 3, 2025
@ianayl
[CI][NFC] Add placeholder workflow file for testing PR #19280 (#19281)
1bba44f 
This PR allows changes in #19280 to be
tested. This PR will be open for merging if changes in #19280 are
approved.
uditagarwal97 pushed a commit that referenced this pull request Jul 3, 2025
@ianayl
[CI][NFC] Add manual dispatch to test workflow for PR #19280 (#19292)
94b20ff 
Forgot to add manual dispatch in #19281
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 15:37 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 15:59 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 15:59 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 16:09 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 16:48 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 16:48 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 18:03 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 18:37 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 18:37 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 18:39 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 19:00 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 19:00 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 20:34 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 20:54 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 3, 2025 20:54 — with GitHub Actions Inactive
@ianayl ianayl marked this pull request as ready for review July 3, 2025 21:00
@ianayl
Copy link
Contributor Author

ianayl commented Jul 3, 2025 •
edited
Loading

Benchmark test runs:

Note: Currently, the build for the benchmarking run doesn't use --no-assertions during configure, as there are build failures when --no-assertions is enabled. This can be for a future PR.

As per Udit's suggestion, I switched the workflow to use Ubuntu 24.04 and that fixed the --no-assertions compilation failures. --no-assertions is working again. See https://github.com/intel/llvm/actions/runs/16078128504 for latest test run of sycl-nightly-benchmarking.yml

@ianayl
Copy link
Contributor Author

ianayl commented Jul 4, 2025 •
edited
Loading

@intel/dpcpp-devops-reviewers Friendly ping, thanks!

Re. prior comment: --no-assertions builds are failing in the current nightly as well

@ianayl ianayl temporarily deployed to WindowsCILock July 4, 2025 16:53 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 4, 2025 17:20 — with GitHub Actions Inactive
@ianayl ianayl temporarily deployed to WindowsCILock July 4, 2025 17:20 — with GitHub Actions Inactive
@ianayl
Copy link
Contributor Author

ianayl commented Jul 4, 2025 •
edited
Loading

@intel/llvm-gatekeepers PR is ready to merge, thanks!

E2E errors are unrelated, as this PR does not touch anything outside of CI workflows files.

@uditagarwal97 uditagarwal97 merged commit b5b9813 into sycl Jul 4, 2025
23 of 29 checks passed
@uditagarwal97 uditagarwal97 deleted the ianayl/move-benchmark-out-of-nightly branch July 4, 2025 21:14
@ianayl ianayl mentioned this pull request Jul 8, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@uditagarwal97 uditagarwal97 uditagarwal97 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ianayl @uditagarwal97